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I. BACKGROUND AND INTRODUCTION 


A. BACKGROUND 


Motivation for this thesis was initially provided by 
Professor James G. Taylor, who wanted to use an existing 
fully operating large-scale combat model as a teaching 
vehicle in a conbat-nodelling course in the Operations 
Analysis Curriculum at th2 Naval Postgraduate School (NPS). 
The original plan was to acquire the ATLAS (A Tactical, 
Logistical, and Air Simulation) model from the U. S. Army's 
Concepts Analysis Agency, convert it for use on the NPS IBM 
360 computer and develop a manual for the setup and running 
or the model as part of the course. Then the attrition and 
movement routines of the model were to be analyzed as a 


formal thesis. 


The project was especially appealing to the author 
because it conformed with a fundamental belief of his, that 
rather than developing new models if an existing model is 
appropriate for a given analysiS it Should be acquired and 
used. The concept is not original but was based on 
recommendations of the Army Models Review Committee. [1] 
Acquisition and use of ATLAS at the NPS would bean 
application of the recommended concept of transferring an 


existing model whenever its use is feasible. 


In late Feb. 1979, the ATLAS model arrived via a 
Magnetic tape, and the author proceeded to execute the above 


plan. Arter the expenditure of about four man-months of 





effort by the author and computer center programmers and 190 
minutes of CPU time, ATLAS was successfully compiled and 
linked. This expenditure of effort was greatly in excess 
of the expected time to complete the task, given that the 
model has been in existence for ten or more years and is 
considered to be "simple" compared with other theatre-level 
models. Why had its transfer required such expenditure of 
effort? 


In reflecting upon the above question, the author 
realized that even though a model may have been used for 
Many years, its transfer can still be hampered by limited 
documentation. This thesis will relate the author's 
experience with ATLAS and his subsequent investigation of 
how documentation limits the transfer of existing models 
between agencies. It will also examine the related 
problems of model complexity, proliferation, credibility, 
transparency, and transferability. Based on this 
examination and the experiences of the author during seven 
years of ‘operation research related asSignments, a new 
concept of documentation to improve the transferability of 


combat models is proposel. 


B. DEFINITIONS 


In research conducted for this thesis the author 
discovered that in modeling the same word can have many 
connotations. For purposes of clarity the author adopted 
the definitions of modeling terms listed in the glossary of 
Ref. 1. This does not imply that they are the only or best 


definitions, they are only used as a point of departure. 


Within the following text there will be references to 


combat models, theatre-level, large-scale, and small-scale 





models. For purposes of the remarks, discussions, and 
recommendations contained herein, the words can be 
considered synonymous. Since the initial impetus for the 
thesis concerned work with a theatre-level model this tern 
readily crept into the author's dicussion. However, all 
these forms of models are just subsets of the concept of a 
combat model. What is addressed throughout the thesis is 
combat modeling. Particular comments referring to one of 
the subsets are just as applicable to combat models as a 


whole. 


Although some remarks are addressed to the DOD, others 
to the Department of the Army, each is applicable to the 
other as well as the other services. The discussions, 
conclusions, and recommendations are equally applicable to 


other complex models as well as modeis in general. 


C. MODELING AND MODELS 


What is modeling? According to Morris [2] nodeling is 
an intuitive process through which an analyst arrives at a 
model. On the other hand, a model 1S an inanimate object, 
an abstraction of reality {3], that is used by the analyst 
to answer questions about some future state of a process. 
This differentiation between modeling and a model is 
necessary to understand the implications of the facts 


presented and the conclusions drawn in this thesis. 


To develop a model, the modeler goes through a process 
of discovery. This process is the trial and error procedure 
in which the designer tries to abstract the key elements of 
reality. Once he has done this and validated his model, the 
logical reconstruction of events leading to the model are 


documented in the context of justification. This logical 





reconstruction has little if anything to do with the actual 
process followed in building the model. No attempt is made 
to verbalize the actual psychological process, the problems 


encountered, or dead ends pursued. 


Over time the initial simple model proceeds through a 
process of enrichment or 2laboration. Through this process 
the model is modified and moved in evolutionary fashion 
toward a more elaborate representation of reality; in the 
process the model becomes more complex as it seeks to 
reflect the complexity of the reality it represents. Each 
time the model is enriched the reasons why and how it is 
enriched should be documented. For a discussion of the 


modeling process see Ref. 2. 


D. INTRODUCTION 


Since ancient times military planners have used wargames 
to investigate various aspects of possible future military 
operations. Historical development of war games can be 
found in Ref. 4 or other histories of war games in the open 
literature. The oldest known form of the wargame is a 
Hindu chess-like game called "Chaturanga". Modern war 
games had their beginning in 1664 thru the games developed 
by Christopher Werkmano called military chess. In the 
twentieth century the greatest proponent of war games till 
the end of WWII was the German Army. Through the 
development of the digital computer during the war a new 
facet of wargaming became possible. The use of the computer 
greatly reduced the time and effort required to conduct a 
war game. Computer assisted or computer run war games 
provide a means of gaining insights and experience in 
Military problem solutions. [5] These computer war games 


help evaluate n2w weapons systems, study current and 
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proposed military organizations and investigate the possible 
outcomes of future conflicts given particular weapons, 
organizations, tactics, and enemy forces. The basis of such 
computer wargames is the combat model. Although there are 
Many military applications of combat models, this thesis is 
primarily concerned with those applicable to military 


Strategy and force planning. 


Military strategy and force planning has matured since 
WWII. Prior to WWII nilitary technology evolved slowly; 
those responsible for strategy and planning could easily 
gain all the necessary knowledge about the relationship of 
military forces and weapon systems to national security from 
books or personal experience. Within one lifetime the 
amount or technological change was not sufficient to render 
experience invalid. The military diðđ not plan on 


technological change; it merely adjusted to it. 


During and subsequent to WWII, the rate of change of 
Military technology began to increase in an almost 
exponential manner. A lifetime of experience could became 
obsolete in a few years; now mere adjustment to change was 
not sufficient, the military had to plan for change. This 
revolution in technology was incubated and nurtured by the 
advent or the digital computer: to keep pace with 
technology and its impact on strategy and force planning the 
digital computer was adopted as a planning tool. With it 
the means to assess and adapt technological change and 
incorporata it into military strategy and force planning was 
possible. [6] 


Modern day force planning has become largely an 
analytical process that necessarily employs the digital 
computer. The digital computer has been incorporated into 
Many of the myriad aspects of military planning and decision 


making in order to provide a scientific basis to these 
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activities. In this thesis, the computer is considered only 
as a computational aid for combat models. The outputs of 
these models are used as aids to decision makers. Strategy 
and force planning depend primarily on large-scale combat 
models, which due to theic complexity require the use of 
digital computers. Mod2ls can be of various types or 
classes; there 1s no agreed upon system of 
classification. Discussions of classification of combat 
models are contained in the literature. {3,7 8] This study 
is concerned with large scale computer simulations and 


analytical models as defined in Ref. 3. 


Ee SUMMARY 


This chapter has discussed the background of this 
thesis, and it has provided some basic definitions and a 
general history of combat modeling to the 
present. Subsequent chapters will now address the following 
important aspects of models used for defense planning: 
@eeticism and credibility; proliferation, complexity and 
transparency, and requirements to transter a model. Next 
a common element of each chapter, lack of adequate 
documentation, is identified as a contributing cause to each 
of the conditions discussed. This is followed by a 
proposed concept of documentation that will alleviate many 


of the problems discussed in this thesis. 
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II. CRITICISMS AND CREDIBILITY OF COMBAT MODELING 


Meee CAUSES OF CRITICISM 


Critics and criticism of defense analysis and its tools, 
of which modeling is just one, are ever present. Criticism 
has not abated Since the early sixties; it continues to the 
present and threatens funding, the life blood of analysis, 
in the Department of Defense. Informed private citizens and 
activitists groups have attacked the propriety of defense 
analysis and DOD decisions based on analysis. The poor 
public image was cited as a contributing factor to the poor 
reception of analysis in Congress. Because of seeming 
inconsistencies in analysis, Congress has become skeptical 
and disenchanted and has questioned the utility of 


analysis. [9] 


According to surveys and studies conducted early in this 
decade [10] activity and expenditures on gaming and 
Simulation peaked in the middle 1960's and were on a slight 
decline since then. These investigations indicated that 
machine simulation had generally been oversold and at that 
time Operations Research and modeling were undergoing a 
critical self-examination. [11] The criticisms were many 
and encompassed a wide variety of cause and effect 
relationships. Those of relevance to this study revolve 
about the inter-related areas of transferability, 


complexity, proliferation and documentation. 
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As the use of combat models increased, so did their 
fre tia lL acceptance and importance, and in turn the 
complexity of these models has also increased. Complexity 
has manifested itself in various forms: inputs, types of 
models, language, detail of actions and conditions 
Simulated, Simulated decision ma king and computing 
machines. Eventually the complexity of existing models, 
poor transparency, and difficulty of transferability caused 
numerous models to be developed that modeled the same or 
very similar combat phenonema. Unfortunately, at this 
juncture in the development of combat models (late 60's, 
emmy. 70'S) criticism ani dwindling credibility occurred. 
This came about because for reasons not easily recognizable 
or understandable to decision makers, models supposedly 
modeling the same combat process under the "same" conditions 


produced different and at time conflicting results. 


Analysis implies rigor and association with the 
Scientific method, yet standards seemed to have waned; 
strict adherance to standards of scientific rigor and 
discipline were less than tenacious. Often analyses and 
models produced had methodological flaws. Often these flaws 
were not discovered until after an analysis had been 
accepted and decisions made. In studies involving models, 
one of the contributing factors to this situation was lack 
of detailed understanding of the model. Other contributing 
factors were the pressure of tine and limited 
distribution. In the rush to meet deadline the quality of 
the work was often sacrificed. By not distributing a study 
or model to other agenci2s the extra set of eyes that can 


see a fatal flaw through an unbiased view were never used. 
The us2 of more than one model has often resulted in 


decision makers being confronted by seemingly contradictory 


results of different analyses using different models 
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addressing the same problem. No wonder that they have the 
impression that the models and methods used by analysts may 
not be very objective guides. Yet most analysts will argue 
that a detailed comparison of models, their assumptions, 
inputs and calculations, Show that the results are not 
really contradictory. Differences in outputs are usually 
the direct result of differences in assumptions and methods 
of processing the input lata. This in turn presents the 
analyst with two challenges: First, he must recognize and 
understand these seeming contradictions, and then he must 
resolve and convey to the decision maker these 
differences. Dr. Ae eur B. Payne, former Deputy 
Undersecretary of the Army for Operations Research speaking 
from the point of view of the decision maker who tries to 
draw valid conclusions from analyses that use such models, 
has argued that he has frequently seen such apparently 
contradictory results coming from various models addressing 
essentially the same problem. Furthermore, the decision 
making process in large organizations is such that the 
detailed comparison , if it is ever done , and resolution of 
seeming conflicts usually does not reach th2 decision 
maker. Hence, the decision maker is left with the 
conclusion that simulation results are not consistant and 


therefore of dubious reliability. [12] 


Fontradietory results of combat models is a factor cited 
by Huber {12] that has caused the very credibility of combat 
modeling to be questioned. Under ideal conditions a model 
Should be directly connected with a continuing experimental 
program and should reasonably relate to other models that 
simulate the same or similar processes. The user must be 
especially watchful in this respect because the combat 
process does not easily lend itself to establishing a 
continuing experimental program. Many combat models are 
neither built nor used with any forethought given to their 


connection to other models. Each generaily turns out to be 
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a totally independent data generator. This precludes any 
meaningful experimental feedback in the on-going prediction 
process and results in a mass of unrelated and often 


contradictory data generated by many models. 


The Army Models Review Committee report was the seminal 
publication on theatre-level combat models for the 
seventies. From it sprang a decade of proposals and counter 
proposals concerning theatre-level combat models. However, 
if one scrutenizes the literature of the previous decade a 
feeling of deja vu surfaces. Many of the ideas and 
arguments sounded in the seventies are in the literature of 
the sixties. For an example of the problems foreseen and 


Warnings given, see Ref. 13. 


Some MPORTANCE OF ASSUMPTIONS 


One of the key facets of any model is the assumptions 
Nat gO into its development. The importance of 
assumptions whether in models or otherwise was recognized 
early in the development of systems analysis. When Mc 
Namara was Secretary of Defense, a continuing effort was 
made to insure assumptions incorporated into models were 
both explicit and consistent. Whether comparing force 
structure or strategy, Mc Namara considered it possible to 
select assumptions that will make any proposed weapon systen 
or Organization look optimal. [6] Likewise, experience has 
shown that there is no single "rignt" set of 
assumptions. There exists an almost infinite set of 
assumptions each more or less defensible. What 1S important 
is that the assumptions used in various submodels of a model 
are consistent. A model should not operate with one set of 
underlying assumptions in one submodel, while another 


Submodel operates with a fundamentally different set. 
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Because there is no emperical data concerning modern 
large scale combat, analysis is [relied upon to produce 
insights into the effects of existing or proposed weapons, 
force structures, and strategies. Using theatre-level 
combat models, the analyst can examine proposed weapons and 
force structures and possible outcomes can be forcast. 
Studies in the DOD [6] have shown that when alternative 
force structures and weapons are examined using different 
models, it is difficult to determine whether differences in 
outcomes are due to differences in weapons and organizations 
Or assumptions and models used. Confusion can be reduced 
if each alternative is examined in a consistent manner by 
using the same model ani assumptions. Differences that 
occur can then be attributed to the basic structure of the 
force options. The results can then be analyzed and 
understood in light of the method of calculation (i.e. the 
model) used and its inherent assumptions. Greater insights 
into the effects of weapon and force structure alternatives 
on combat outcomes can be gained by repeating the experiment 


uSing an alternative model. 


When more than one model is used for the same force 
structure analysis often different results are obtained. 
These different results ar= caused by the different inherent 
assumptions of each model. If these assumptions are known 
informed discussion can take place because differences can 
be resolved through evaluation of the assumptions. If the 
assumptions are valid andi acceptable, then the results must 
be accepted as the logical consequence of the assumptions. 
ASsumptions considered to be unacceptable by decision makers 
can be eliminated by saodifying the model. What quickly 
becomes evident is that a model produces a result based on 
its inherent assumptions; an equally defensible but 
different set of assumptions used in a mcdel will produce 


another result, which may or may not be the same. 
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The DOD is quite aware of this aspect of modeling and 
has emphasized that there is more than one set of 
assumptions that can be used with a given model. It 
requires that all parties to the decision making process be 
aware of all assumptions leading to a result. To achieve 
this goal the DOD has required that all assumptions be 
eeplicit, reasonable, and consistent. This requires 
ferreting out hidden assumptions in models and insuring that 
the model indeed models what it claims to model. FOr 
interesting examples of the importance of assumptions see 
Ref. 6. 


C. MODEL EVALUATION 


Many solutions to the problem of model evaluation have 
been recommended; one concept suggests the use of full time 
dedicated independent reviewers as a way of improving the 
mechanics of quality control. [1] A reviewer provides a 
means through which the analyst's model is scrutinized; the 
assumptions and methodology are checked for internal 
consistency, unwarranted inferences, and celari: y” of 
presentation so that a determination can be made whether or 
not the model is a plausible representation of the real 
world. If it is a large complex model, the methodology 
should be clear, even to the point of sample calculations to 
guide the reviewer and analyst through the algorithms. 
Equally important is that input data and assumptions be 
explicit. If unnecessary proliferation is to be avoided, 
existing models must be made understandable to potential 
users and evaluated in some manner. It should not be 
necessary to create a new model just because an existing 
model could not be transferred or understood due to 


inedquate documentation. 
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In his study, Gass [14] proposes an elaborate approach 
to evaluation of complex models. Therein he highlights the 
need for model implementation and maintenance procedures as 
well as documentation of the model and the total modeling 
process. Suggested documentation of the process includes 
describing model objectives, assumptions, results, data 
sources, recommendations, etc. With such documentation the 
model can hopefully be evaluated and analysts can determine 
whether or not the mod2l is valid for the problem at 
hand. However, Gass found that for most complex computer 
models, organizational exigencies and real-world pressures 
do not enable modelers to develop the necessary 


documentation. 


Gass has stressed the need to validate models at three 
Ns tinct levels, technical, operational, and 
dynamic. Technical validity is an assessment of model, 
data, logical, mathematical and predictive 
validity. Operational validity is an assessment of errors 
and divergences found under technical validity and the 
robustness of the model (i.e. whether or not the model can 
produce bad answers for proper ranges of parameter values). 
Dynamic validity is the method by which the model will be 
maintained during its life cycle so that it continues to be 
an acceptable representation of the real system. It 
includes the process through which the model structure is 
Changed and validated. The ability to accomplish such 
validation will facilitat]a and enhance the use of models by 
anaylsts and decision makers other than those directly 
responsible for the development of the model. Fundamental 


to this process is detailed understanding of the nodel. 
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D. SUMMARY 


If a model is to be of value it must be accepted by the 
decision maker. It is incumbent upon the analyst to provide 
the means of acceptance. Sufficient documentation must be 
provided by the model designer to enable the analyst to 
understand its methodology and structure. Key eo 
credibility is objective evaluation. Documentation must 
provide insights into the assumptions and functioning of a 
model, so that it can be evaluated. Understanding and 
evaluation are complicated by complexity. Complexity and 
insufficient documentation can cause analysts to design new 
models rather than use an existing model. The existence of 
inadequately documented models describing the same reality 
is the basic cause of the criticism and lack of credibility 
of combat models. The next chapter will discuss complexity 


and how and why it contributes to unnecessary proliferation. 
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III. PROLIFERATION, COMPLEXITY, AND TRANSPARENCY 


Unnecessay proliferation is the cause of some of the 
Criticisms of combat modeling. Unnecessary proliferation 
means that a new model is created because inadequate 
documentation precludes the use of an existing model which 
otherwise would be adequate for the task at 
hand. Inadequate documentation 2ither prevents 
understanding of the methodology and structure of the model 
or prevents the cost-effective transfer and conversion of 
the nodel from one agency and computing systen to 
another. The latter condition is the one encountered by the 
author during the conversion of ATLAS. Better documentation 
would have significantly reduced the resources expended on 
the project. Futhermore, in a non-academic environment the 
demands of proceeding with an impending study would have 
encouraged analysts to develop a new model rather than 


Struggle with poor or non-existing documentation. 


Development of a new model can cause analysts to remodel 
a combat process using techniques and methodology that 
already exists; consequently funds are expended without 
advancing modeling. Irrespective of any advancements in 
techniques or improvements in methodology developed in 
remodeling an existing process, a major portion of the 
effort is devoted to redoing the basics. Time and money 
Spent in redoing the basics are lost as far as improving 
models and modeling is concerned. In the DOD (especially 
the Department of the Army) there has been and continues to 
be a shortage of trained analysts. An obvious approach to 
alleviate this problem would be to eliminate any projects to 


develop models that would be redundant and share existing 
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models. However, the ability to share a model is greatly 
influenced by its complexity. 


When examining the practicality and feasibility of model 
Sharing most emphasis has been placed on 
documentation. Huber [8] lists poor documentation and high 
personnel turnover as a prime reasons why models are not 
exchanged. The criterion used to determine whether or not a 
model is transferable is documentation sufficient to allow 
the recipient organization to be able to run the model 
without an Lnordinate amount of decoding and 
deciphering. Kithout adejuate documentation tha model is a 
puzzle to the recipient. Shubick and Brewer [10] found that 
few models exist in a sufficiently documented form that 
would Satisfy commercial firm standards PELOT to 


Mrstribution to their clientle. 


A. PROLIFERATION AND HOW TO REDUCE IT 


An area examined by the Comptroller Ganeral in 1974, 
[15] was sharing of computerized models. Models developed 
for specific purposes by one agency can often be used by 
other agencies for similar purposes. Applicability of 
models to new situations depends on their accuracy, purpose, 
Validity, availability of sufficient documentation, and 
capability of the using analyst. The 1974 study surveved 
242 models that had a combined cost of over thirteen million 
dollars. An attempt was then made to obtain documentation 
for about one hundred randomly selected models deemed to 
have use at more than the originating agency. Information 
explaining the purpose, mathematical formulation, and 
operating instructions were not available for approximately 
one-third of the models. The survey identified the 


primary complaints of model users (programmers and analysts) 
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as: operating instructidns not available or not clear, 
hence, compounding the already difficult problem of 
preparing a model for use on a different computer; 
mathematics of the model not clearly explained, hence, 
restricting the understanding of the logic, capabilities and 
limitations of the model; sampl2 inputs and outputs 
nonmatching or nonexistent or do not correspond with the 
sample data in documentation, hence, verification and 
validity of the model difficult to determine; flow chart 
explaining logic not provided or not current, hence, complex 
Subroutines not easily understood. The investigation 
determined that benefits can be obtained by sharing computer 
models, however, before models can be shared adequate 
documentation must be prepared. Such documentation enabled 
the acquiring agency to istermine whether a model met its 
needs and was the primary factor to successful conversion 
and operation of the model on a different conputer. The 
potential for a cost-effective transfer is severaly limited 
in the absence of adeguate documentation. When such 
documentation is available to potential users of an existing 
model great savings are realized. An exanpl2 was the 
transfer of a complex communications traffic analysis nodel 
from an Air Force agency on the West Coast to the Systems 
Development Command at Hanscom Air Field in Bedford, 


Massachusetts. [15] 


Joint usage of existing models not only increases the 
availability of trained individuals to do further research 
and analysis, but it reduces the opportunity that different 
factions of the same organization are working at cross 
purposes. Conflicting concepts and proposals are necessary 
to vitalize an organization and make it viable, but 
developing a conflicting position that could be resolved 
prior to the expenditure of great sums of money and 
analytical talent is a waste. The sharing of a model or 


models between two conflicting agencies allows each to 
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understand the underlying basis for their respective 
positions concerning an analysis of a decision making 
Situation. While sharing may not resolve the conflict it 
certainly will preclude the expenditure of funds in the 
independent acquisition of information that is already 
available through the medium of an existing model. Sharing 
has eliminated retracing steps already taken and dead ends 
already discovered when applied in other fields , applied to 
combat modeling sharing will provide these minimal returns 
and has the prospective of providing even greater 
returns. If the basics are not reprocessed then nore time 
and money is available for modifications to enhance the 
capability of an existing model, correct known deficiencies, 
or identify suspected deficiencies. Sharing has promise to 


improve the economies of modeling. [16] 
Before sharing can be achieved certain basic conditiions 
must prevail. Five necessary conditions [17] to model 


sharing have been found. They ars; 


(1) a computer able to use the program with minimal 


Modification; 
(2) an adequate facility to run the model; 
(3) adequate documentation of the original model; 
(4) sufficient analysts with technical competence; 


(5) formalized arrangements for sharing costs and 


responsibility for costs and coordination. 
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poeeeiae EFFECTS OF COMPLEXITY ON TRANSPARENCY 


Complexity is a dichotomous issue in itself. Gass 
perceives an increasing use of more complex models [14], 
While Hardison, the Deputy Under Secretary of the Army for 
Operations Research at the 15th Annual US Army Operation 
Research Symposium called OE less complexity. 
Disatisfaction of both senior military and civilian decision 
Makers with complex models and studies was emphasized by 
Hardison. [18] These decision makers are convinced that 
Army models and the studies they support are too complex, 
elaborate the obvious, belabor needless details and overlook 
key issues. Timeliness is also affected by 
complexity. Failure to delimit results in failure to meet 
schedules which causes something to be sacrificed; often, in 
the case of models that something is adequate 


documentation. 


A corollary of the complexity issue is that of 
transparency. If all the interactions of a model are to be 
understood by both the analyst and the decision maker it 
must be structured and programmed so that its methodology is 
easily understood. A model that fulfills this requirement 
is said to be transparent. At the 35th Session of MORS a 
leading cause of the general disenchantment with 
theatre-level models in recent years was attributed to a 
lack of transparency in most models. The proposed 
resolution to this problem was to include in the model only 
those interactions and factors that can be shown to 
influence the outcome. This in combination with 
Mathematical fornulation that is as simple as possible 
should produce the desired transparency. [19] Yet, at this 
same MORS session A.H. Cordesman OASD (Intelligence) in 
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discussing theatre-level models remarked toit models 
currently being develop2?d go into unnecessary levels of 
detail in ways which seriously limit their value. This is 
partially caused by intermediate managers and decision 
Makers requesting particular attributes be modeled. At 
times modeling efforts deviate from the maxim that only 
"essential" variables be nodeled. As Morris [2] suggests, 
the purpose of a model is to include only thos? variables 
that characterize the process being modeled. At present 
diametrically opposed forces exist; while expounding the 
need to maintain models at a simple transparent level 
current modeling efforts go into details that detract from 


transparency. 


mesimple solution to the transparency-complexity problem 
may not be easily obtained. In spite of the professional 
rhetoric to the contrary, Gass [14] has found an increase in 
the use of complex models at all levels of government and 
mmaistry. He attributes this to better trained analysts and 
the development and refinement of methodologies. Although 
simple models with readily understood assumptions, 
relationships and structure are preferred, Gass conténds 
that decision making problem environments representative of 
the Federal Government sphere cannot be realistically or 
logically contained by simple models. Furthermore, senior 
decision makers generally do not possess 2 detailed 
understanding and appreciation of the methodologies employed 
in the various models employed to assist them in the 
decision making process. What is needed is a method 
through which the use and interpretation of the outputs of 
models by senior decision makers is facilitated. A model is 
usable only if it is understood and plausible to analysts 
and decision makers. They (particularly the decision maker) 
Must be given the opportunity to explore the use of tae 
model, become familiar with its predictions, and 2xamine the 


relationships and assumptions implied by the model. In 
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actuality the demands on their time generally preclude 
decision makers being involved at the detail levels implied 
above. Therefore, it is incumbent upon analysts to evaluate 
the models they use so that they are in a position to 
recommend to the decision maker to use or not use the 
outputs of a particular model. This implies intimate 


knowledge of the essence of a model. 


C. SUMMARY 


To reduce unnecessary proliferation and reduce costs the 
Comptroller General has recommended model sharing when 
feasible. Sharing a model requires: 

a ability to use it with minimal conversion; 

a adequate facilities to run the model; 

s sufficient competent analysts and programmers; 

a adequate documentaton; 

a formalized arrangements for cost sharing and 


Coordination. 


Findings indicated that the great complexity of 
theatre-level models coupled with rapid turnover orf 
personnel has resulted in models being used as "black boxes" 
With neither the computer technicians that run the model nor 
the analysts knowing explicitly what or how the model 
Operated on the input data to provide the final results or 
Output. Hence, the analyst was unable to adequately explain 
the results to the decision maker; with each occurence the 
credibility of modeling diminished. Concurrently models 
had proliferated to such a degree that the turnover of 
personnel exacerbated an already critical personnel shortage 
Situation. Amelioration requires reduction of the number of 
models in use and detailed justification before developing a 


new model. [1} Reduction or the minimization of the growth 
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of the model inventory implicitly requires that models be 
easily and quickly moveable from one using agency to 
another, i.e. posses transferability. Key to the resolution 
of the problems of complexity and transparency is 
documentation. 

Complexity may be an unavoidable recourse of combat 
modeling because of the demands of managers and decision 
makers. Unless complex models are sufficiently documented 
to make them readily understandable and usable, analysts 
Will create anew model. Rather than creating new models 
an atmosphere conducive to the sharing of models should be 
incouraged. Requirements to transfer a model are discussed 


in the next chapter. 
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The Review of Selected Army Models report [1] proposes 
that the number of models retained by the Dept. of the Army 
be reduced and that existing models be used where possible 
before a new model is commissioned. To properly evaluate an 
eXisting model a method for analyzing and verifying a 
candidate or candidates from existing model resources must 
be established. The more complex a model is the nore 


difficult is this analysis and verification. [14] 


Use of an existing model requires understanding of 
potential problems due to model design as well those 
problems expected to be encountered during transfer and 
execution. Design problems can include lack of adequate 
sub-models, failure to consider key variables, inaccuracies 
Black of validation, computational difficulties, and 
inconsistent hidden assumptions. Problems during exection 
can be those of irrelevance, inadeguacy of output, 
inappropriateness of assumptions, lack of connection to 
other models and results, statistical and extrapolation 


Seecicultics. 


A. METHODOLOGY FOR ANALYSIS OF EXISTING MODELS 


Before an existing model can be used it must be analyzed 
by the potential user. Examination of the literature for a 
methodology for conduct əf an analysis generally produces a 
concensus. Such methodologies center on five general areas. 


[20] These are: 
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(1) inputs and outputs, i.e. the global structure of the 
model; 


(2) the basic causal relationships assumed between 


Variables; i.e. the local structure of the model; 


(3) the detailed logic of the model; 


(4) the numerical values of the data, and 


(5) the time and resources reguired to exercise the 


model. 


Additionally, an analysis should consider any 
experimental studies that allow comparison of the model 
predictions with the real world, other models or with the 
intuitive beliefs of the decision maker that will ultimately 
be presented the outputs and their interpretation. Such 
previous studies are useful in evaluation of a model for 
application to new problems or Situations. Unfortunately, 
so far as combat models are concerned, comparisons with real 
world results are extremely limited. When such data is 
available (e.g. WWII and Korea) it is sparse, subject to 


conflicting interpretation, and of questionable accuracy. 


(21) 


A detailed examination of the glopal structure of a 
model can forthrightly answer the basic question of such an 
analysis. Is this model capable of examining the problem 
at hand? The potential user is interested with whether the 
outputs measure the desired quantities and or gualities, and 
whether the desired inputs can be entered into the model in 
the form in which they are available. Using an existing 
model is not cost-effective if available input data must 
undergo a costly and time consuming conversion. There must 


be provisions in the mod2l structure to allow changes in the 
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input data to reflect changes in the combat process under 


examination. 


The local structure 1S examined to expose important 
causal influences which may have been omitted for ease of 
computational effort or other reasons. This step is closely 
related to the abstraction process in the design of the 
model. It is most dependent on the detail and completeness 
of available documentation. It is critical because the 
importance of a causal connection can be quite subtle but 


very pervasive. 


Examination of the detailed logic of a model identifies 
the hypothesis upon which the model is based. It reveals 
extent to which it 1s based on historical and experimental 
field data. It is another indicator of the appropriateness 
of the model to solve the problem at hand. This step also 
reveals the presence of inconsistent assumptions or 


inappropriate assumptions. 


Finally, analysis of the time and resources required by 
the model provides the means through which the costs 
associated with the use of the model can be determined. To 
make rapid and accurate 2stimates the description of the 
model must provide explicit information of time requirements 
to gather and prepare required inputs as well as the time to 


execute the model given the inputs. 


If models are to be truly transferable between agencies 
the above analysis must be comleted prior to any attempt to 
transfer a model. Accurate and complete documentation must 
be available Tr this is to be conducted. Such 
documentation will be available only if it is prepared 
concurrently with the development of a new model and in 
concert with any modifications made during the life of the 


model. 
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Be. POTENTIAL PROBLEMS IN EXISTING MODELS 


To adeguately analyze an existing model the analyst must 
be familiar with potential problem areas. Farrell [209] has 


suggested that the analyst consider: 


(1) the adeguacy of submodels, i.e. do they model what 
they purport to model; 


(2) whether key variables were overlooked during the 


process of abstraction; 


(3) the possibility of inherent inaccuracies; 


(4) possible computational difficulties; 


(5) whether the mod2l has inconsistent and inapproriate 


assumptions; 


(6) possible inadequacy of output. 


Among the problems of design can be found cases where 
the general model structure is adequate but reasonable 
submodels are not available. This is particularly true of 
sub-models involving Simulated tactical or Strategic 
decision making. Farrell (20] has indicated that most 
diagrams and flow charts of such sub-models do not reveal 


the sub-model inadequacy. 


In designing a model the first and most difficult step 
is insightful abstraction by the modeler or molelers. To 
abstract the real worli into a representative model, key 


Variables, their basic causal relations and interpretation 
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must be incorporated into the model. This step is very 
difficult to verbalize since any description of the process 
to be modeled will necessarily incorporate the results of 
abstraction in the elements of the description. This step 
is imaginative, creative, and complex; it is imperative that 
it be documented at the time of abstraction. Once a 
process has been modeled and time passes the explicit steps 
and reasoning thru which the abstraction was made cannot be 
adequately described by the modelers. Failure to consider 
key variables occurs at this almost invisible step of 
abstracting a process of the real-world into a model or 


sub-model. 


In the review of an existing model it is these almost 
invisible steps of abstraction that must be thoroughly 
examined to insure all key variables are included in the 
model. Key variables can also be excluded because of lack 
of adequate sub-models or computational problems their 
inclusion or manipulation would precipetate. The reviewer 
or user of an existing model must carefully search for key 
variables that have not been modeled or are simply thruput 


in the model. 


Another pitfall hidden in the design of existing models 
is inaccuracy or lack of validation. Often because of 
computational difficulties known experimental results have 
not been included in a particular model. This occurs most 
often as a result of exigencies on tne modeler or modelers 
at the time the model is created. Exclusion of such 
experimental data results in inaccuracies. Lack JE 
experimental support for portions of military models is 
another common cause of inacuracies which cannot always be 
avoided. Likewise, inadequate debugging of the model can 
be a reality. This is not serious as long as these 
inaccuracies are known. Unfortunately, these facts can 


escape a potential user if a careful review of the model 
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design and its use 1S not made. 


A thorough review of model documentation can reveal 


computational difficuüuities if the documentation is 
adequate. These difficulties usually involve either 
megorithmic imprecision OG excessive date storage 


requirements. These limitations should pe revealed by the 
documentation to the potential user so that proper use of 
the model can be made, unexpected or excessive costs are not 


incurred or another model can be selected. 


The bane of analyst using an existing model is 
inconsistant hidden assumptions. Such assumptions included 
as part of the overall model can be at odds with those of 


sub-models, the data base, and data generating routines. 


The most pervaSive error awaiting the unwary user of an 
existing model is the inappropriateness of inherent 
assumptions. This potəntial error is the most difficult 
to detect when using a pre-designed combat model, since the 
user is generally not well versed in the process through 
which the abstraction of the real world was male. During 
the abstraction fundamantal assumptions concerning the 
mature of the combat process as well as assumptions for 
computational reasons were made. Only through careful study 
of the line of reasoning followed at the creation can a 
would-be user become familiar. with these assumptions. Care 
must be taken so that not only the explicit but also the 
implicit assumptions are understood and their effect on the 
combat process being modeled is 
comprehended. Unfortunately, the reasoning and logic 
followed in creation of a model are not included in current 


documentation. z 
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If there are potential problems awaiting the unwary 
analyst in the design of an existing model, these are just 
the forebodings. of greater problems associated with the 
actual exercise of the model. Foremost of these is 
irrelevance, that is caused by either attempting to use a 
model on a problem for which it was not designed or a 
failure to understand the problem thoroughly enough to 


select an appropriate molel. 


Concomitant with irrelevance is inadequacy OE 
output. This results when useful information is inherent in 
the modei itself put actual calculation and or display does 
not exist. Causes of such conditions are selection of an 
improper model or lack of full understanding of the 
fmesicacies of the model. Proper and adequate 
documentation and careful perusal will ameliorate the 


situation. 


In complex simulations, both situations and key 
parameters will b2 variei and the results examined. The 
number of unique situations examined is limited by resource 
and time constraints, because of this the potential user 


must be sure that required outputs are readily available. 


Adequate descriptions of the output of any random 
process are dirrieult to achieve under the best 
conditions. Under time constraints descriptions of the type 
Of output provided by a model as well as how the outputs are 
collected is critical. Without it, the user cannot 
correctly interpret the results obtained. When available 
time and resources preclude simulating ali situations the 
problem af extrapolating or interpolating between the 
particular situations modeled arises. There is no adequate 
general method for surmounting this problem; the problem is 


less severe the less complex is the model being used. Any 
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statistical difficulties encountered only compound the 
problem. Surmounting this problem is a function of the 
ingenuity of the analyst and the detail of the documentation 
available to him. In the case of theatre-level combat 
models obviously, neither all situations can b2 simulated 
nor are there adequate historical or experimental data with 
which to compare the results obtained. A thorough 
accurate and consistent interpretation of the outputs of 
such a model is highly dependent on the analysts intimate 
familiarity with the mathematical structure within the model 
aS well as interactions between the various input and output 
data. Such intimacy is obtainable only through available 
documentation if the analyst uses an existing model designed 


by someone else. 


BE SUMMARY 


Transfer of existing nodels between agencies is one way 

of readucing proliferation. Prior to such a transfer a 
potentionally usable nodel must be analyzed for 
applicability and approriateness. Such an analysis should 
consider: 

» inputs and outputs; 

s basic causal relationships; 

a model logic and structur:3; 

a available data and required data; 


» time and resources required. 


Many potential problems are contained in an existing 
model, among these are: 
a submodel inadequacy; 
s key variables excluded; 
a inherent inaccuracies; 


a conputational difficulties; 
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e inconsistent assumptions; 
e inappropriate assumptions; 


» inadeguate output. 


The ability and to what degree model analysis and 
consideration of potential problems can be accomplished is 
determined to a great degree by available 
documentation. Adequacy of documentation is considered in 
the next chapter. 


37 





A. DEFINITIONS AND BACKGROUND 


Program documentation is a collection of information to 
explain the design, development, and maintenance of the 
program as well as purposes, methods, logic, relationships, 
capabilities and limitations. [5] Except for the simplest 
programs it is difficult, if not impossible, for someone 
other than the originator to determine what is supposed to 


be accomplished by just reading the program code. 


Documentation is necessary for: planning, programming, 
managing, operating and evaluating models. It is absolutely 
essential for: quick and effective changes; use of the 
model by programmers and analysts other than the 
originators; understanding of what is being done; 
interagency program sharing; verification of proper model 
operation. Through adequate documentation seconiary users 
gain an understanding of a model and thus the model and its 
outputs are rewarded a level of credibility. It is vital 
if secondary users are to be able to run the model and make 
necessary modifications of the program. This restrains 
proliferation and duplication which can result in major 
savings; besides It tempers an already complex 
environment. Unfortunately, current documentation practices 
are such that the documentation to facilitate the use of 


existing models generally does not exist. 
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B. CURRENT DOCUMENTATION PRACTICES 


Although documentation is an aspect of computer 
programming that was recognized from the inception of 
automatic data processing as being critical to successful 
programming, it has been and still is a major problem 
area. Unfortunately, for various reasons including time and 
fiscal constraints as well as lack of definitive guidance 
concerning requirements and standards, the bulk of the work 
effort concerning documentation has consisted of unfulfilled 
requirements. Notwithstanding the fact that programming 
has existed since the inception of ENIAC in 1944, the lack 
of adequate documentation received major attention in 
studies concerning models and simulations as well as 
investigations by the Comptroller General of the Army in tne 


late sixties and early seventies. ( 10,15] 


Irrespective of the aforementioned studies the problem 


of inadequate documentation persisted and was the subject of 


ancther investigation əy the Comptroller General in 
1974. [15] Over seventy federal installations in the 
continental United States, Europe, and Asia were 


surveyed. These included selected DOD Agencies as well as 


those of each of the arm2d forces. 


The study cited several problem areas attributable to 
inadequate documentation. Increased cost of operations was 
high on the list. Because of inadequate? documentation use 
of operating programs is hindered since current operators do 
not fully understand how and what is being done ina 
program. Therefore, when unexpected outputs are obtained 
as difficult if not impossible to determine their 
validity. Equally perplexing is inadequate documentation of 
Subsequent modifications incorporated into the model. In 


many cases without adequate documentation it was impossible 
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for a new analyst ani or programmer to use oc modify an 
existing progran. Ultimately such models have to be 
completely rewritten or the time required to make then 
usable was greatly in excess of the time required when 
adequate documentation was available. In a cited example 
inadequate documentation caused an agency to spend over a 
year to determine how a particular complex model 
operated. Another example indicated the difference of six 
man-months of work incurred because of the lack of 
sufficient documentation. Although the Comptroller General 
found it difficult to determine the aggregate cost of 
increased operating expenses due to inadeguate 
documentation, he did indicate it was high because of tae 


number of cases uncovered. 


Lack of sufficient documentation was the major cause of 
the problems encountered by the author during the conversion 
of the ATLAS model. ‘When machine differences required 
changing the program code there was minimal guidance as to 
which sections of the code corresponded to’ particular 
functions described in the user's manual. 22] Also, 
details of the mathematical structure of ATLAS are not 
contained in the manual. For details of the model 
structure references arə made to a models manual [23] for 
the predecessor of ATLAS. The extent to which the structure 
of this model has been incorporated into ATLAS is not 
explicitly stated. The situation is further complicated by 
the fact that pertinent assumptions basic to the model 
formulae are not in the models manual; they are in the 
user's manual listed in a haphazard manner. Since the 
models discussed were originally designed for the 
predecessor of ATLAS there iS no assurance that changes to 
the model formulas were not made during the evolution of 
ATLAS. This Suspicion is enhanced by the fact that in 1973, 
five years after the design of ATLAS, a significant 


programming error was found. [24] 


40 





There are a myriad of reasons why model documentation is 
inadequate. One of the primary causes is that documentation 
guidelines and policies are developed by individual Federal 
de partments and agencies. At the highest levels the 
guidance is necessarily general, as it moves down the 
organization further more explicit implementing directives 
are provided culminating in directives issued by the 
developing agency. Hence, some documentation is brief and 
Simplistic; other documentation is detailed, voluminous and 
complex; neither may prove to be adequate. Adequacy is 
determined by the ability of other than the originators to 
use and understand the model. The Comptroller General found 
that even when guidelines and standards were prescribed 
managers of modeling projects failed to insure 
compliance. The type and content of documentation is often 
decided by computer technicians or ADP operators. [15] 
Shubik [17] has cited this practice as unprofessional since 
ambitious programmers have been known to change coding in 
the pursuit of computing efficiency without making note of 
the fact. 


An examination of 264 model documentation packages at 10 
California installations revealed none fully complied with 
the agency standards and most were incomplete, inconsistant, 
and inadequate. [15] In most cases programmers determined 
what documentation to prepare based on their own 
judgement. Managers responsive for developing models 
indicated that the reason standards were not adhered to was 
because of time contraints. Completion dates frequently 
were given precedence over preparation of adequate 
documentation. The desire to complete a model and get it 
operational by 2 given date overrode the need for 


documentation. 
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An aspect of managerial responsibilities concerning 
documentation is that it be kept current. Even if a model 
is initially adequately documented it will eventually be 
modified and its documentation must be likewise 
updated. This is a major problem because it requires 
diligence to update documentation. Siven time and resource 
constraints and the exigencies of the decision making 
process it is a task that can easily be put off since the 
model will work without such documentation. The problen 
comes later when the personnel that modified the program 
become involved with other demanding problems, leave or are 
transferred. Later the documentation is difficult to 
prepare because the reasons why or how the modification was 
made become unclear or those that knew what was ione are no 


longer available. 


There are numerous comment cards in the ATLAS program 
that indicate changes were made. However, there is no 
formal documentation, with one exception, to indicate what 
or how these changes were made. Informal documentation 
provided was minimal and superficial and did not address all 
the indicated changes. The one exception was a formal 
document [25] prepared in 1974, which discussed improvements 
in the treatment of barriers and personnel replacements. A 
global variable and subroutine listing were provided but 
these were not up to date; had they been current the 


conversion would have been facilitated. 


Poor or nonexisting documentation persists irrespective 
of the efforts of the Comptroller General and the Department 
of Defense. Inspite of the identification of this problem 
early in this decade [1,10,15] its presence continues to be 
a problem at the close of the decade. [3] The continuing 
lack of detailed documentation that enables an analyst to 


understand what goes on inside a model is cited by Shupack 
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[26 ] in lo7 5G mecsceeceelianiting factor in the use of 
theatre-level combat models. Although there was a 
noticeable improvement, he found the level of documentation 
of IDAGAM not sufficient to insure the easy and proper use 
of the model without supplementing the available 
documentation. Without adequate documentation the analyst 
1s apt to make erroneous conclusions regarding the processes 


occurring within a model. 


Although documentation problems persist genuine attempts 
to resolve the problems have been made. Current combat 
modeling efforts at the Naval Postgraduate School are not 
only using languages especially designed for simulation but 
the documentation methods employed enhance the transparency 
of the model. For exampl2s see Ref. 27, 28, 29. Other 
agencies have also made inroads toward improving the 
adequacy of documentation. See Ref. 10 31, 
32. Irrespective of these improvements the author believes 
a vital aspect of documentation is being overlooked. This 
aspect and appropriate recommendations are set forth in the 


next chapter. 


C. SUMMARY 


Studies as well as intuition reveal that to understand 
the workings of a complex model an analysts requires a 
detailed explanation of the calculations and data 
Manipulation performed by a sannlarrons. | Emplicit and 
explicit assumptions and inputs must be known if an analyst 
is not going to use a model as a black box. A detailed 
knowledge of the variables, subprograms and their 
relationships must be acquired if a model is to be 
transferred from one using agency to another. Rarely will 


both organizations posses the same brand of data processing 
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equipment, more likely than not there will be great 
dissimilarities. Conversions from one machine to another 
requires that changes be made to the model. To change the 
model the programmer and or analyst must know the effect his 
change will have not on only the particular line of code he 
is changing but throughout the program. Analysts must not 
only k now how a change will affect the physical 
minipulations and computations of the various parts of the 
model but how it will interact with the implict and explicit 
assumptions of the model. To gain this level OE 
comprehension of a model designed by someone else the 
analyst and programmer of the gaining organization must 
consult the documentation provided with the model. 

Without such documentation analysts have chosen to construct 
a new model. Unfortunately, the general concensus of the 
investigations was that documentation was of dubious quality 


and generally inadequate. [1,10,15] 
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VI. PROPOSED HIERARCHY OF DOCUMENTATION 


The discussion of who the user is, the role of the 
officer analyst, and the documentation concept that follows 
is based on seven years experience in operations research 
related asSignments, discussions with analysts, programmers 
and students, and problems encountered during conversion of 
the ATLAS model. The author has drawn upon the above 
sources and selected literature (4, 15, 17, 33, 34, 35] to 
propose a new concept in model documentation. This new 
concept is intended to refine and supplement Current 
documentation methods. Implementation of th2 proposed 
concept will hopefully fill a void that currently exists and 
Will greatly improve the transparency and transferability of 


combat model. 


A. THE USER AND THE ANALYST 


Before proceeding further it is necessary to define the 
often referred "user". It is one of the more vague terms 


associated with combat modeling. 


Now, who is the user? The user is the study director 
and/or decision maker. These individuals need documentation 
that provides an overview of the model, with indications of 
its capabilities and limitations and general applicability 
to the problem under study. The details of the conceptual 
basis of the model and how and what information can be 
provided along with an evaluation of its suitability should 


be provided by the analyst. 
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The analyst is the manipulator of the model. He is 
responsible for using the model to support the study 
objectives. This means he is responsible for the collection 
and insertion of input data and interpretation of the output 
data. If the model is small, he may do the coding; or if it 
1s complex, a programmer may assist him in the coding 
process. Any interpretation of how the model represents 
reality is the analyst's responsibility. To fulfill this 
responsibility he must be familiar with the conceptual basis 
of the model, its underlying assumptions, how the model was 
transformed into the program code, and how the code operates 
to present the process modeled. The analyst must know if 
any changes to the conceptual foundation of the model 


occurred in the process of programming (coding). 


There is a reluctance on the part of many military 
analysts to gain an intimate understanding of a complex 
combat model. Exingenciss of the organization are some of 
the prime reasons. In the daily demands to manipulate a 
model and to produce results there is insufficient 
allocation of time to study tae inner workings of the 
model. Reliance is placed almost exclusively on the user's 


Manual to explain the results produced. 


Some analysts (this is particularly true of the officer 
analyst) refuse to gain detailed understanding of the inner 
workings of a model because they do not perceive that to be 
their function. Their perception is that they are the user 
and do not require that level of detailed knowledge. Others 
hesitate to learn or become well versed in the functions of 
a particular model because they fear such expertise will 
label them and hence limit the scope of their future 
assignments. In discussions with officer analyst students 
and practicing officer analysts, the author found these 
attitudes to be quite pervasive. Many considered detailed 


knowledge of how a model operated to be in the realm of the 
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programmer's responsibility. Little or no consideration was 
given to the fact that most programmers have little or no 
Military background and cannot possibly relate the 
machinations of the program code to the combat process. The 
programmer views proper operation in terms of proper 
execution of programmed instructions. Whether or not 
instructions properly depict the realities of the combat 
process are beyond their scope of interest. They have 
neither knowledge, inclination or experience to evaluate the 
fidelity of the code. 


Attitudinal attributes of military analysts can be 


understood in Tight of their rapid turnover in 
assignments. Rapidity of reassignment discourages tne 
desire . to gain detailed knowledge of any particular 


model. Most likely they will be reassigned shortly, to 
other type duties. If subsequent assignments deal with 
combat modeis, most likely, it will be a different 
model. Any time expended in the study of details of the 
model assoclated with the current assignment is considered 
of minimal value. When an effort 1s made to understand a 


model it is often frustrated by inadagquate documentation. 


Probably the most frequent attitude seen in officer 
analysts is the one associated with the military psyche, 
that of being a generalist. This attitude is the result 
of the total experience of the profession; it has been the 
way to reach success in the past, and many believe that it 
Still is the way to success. Although there has been 
considerable effort to change this nind set, the example of 
the iast few centuries is difficult to overcome. Compared 
to the existence of the nilitary profession, the experience 
of the military analyst is of recent vintage. Only 


favorable experience will provide the impetus for change. 
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The transfer of models between agencies will contribute 
favorably toward convincing analysts to gain detailed 
knowledge of combat models. Ii they know that the effort 
expended now can be used at a later time in some future 
asSignment, the effort will be considered worthwhile. But 
adequate documentation is a prerequisite of such 
transferability. Hence, there is an interdependency 
betweeen documentation and convincing military analysts to 
gain a detailed understanding of combat models. If they can 
see value in the expenditure of tae time and effort, they 


Will be willing to make the commitment. 


Shubik [16] has found that the mathematical nodeler and 
the person who understands the reality the model attempts to 
abstract are not necessarily the same person. The combat 
process is best understood by senior military decision 
makers who are generally unable to translate it into the 
appropriate abstraction and who generally desire greater 
detail than is necessary. A good model is one that is able 
to abstract and describe only that which is relevant to the 
problen under investigation. On the other hand the 
Mathematical modaler is frequently an individual who 
generally lacks the experience and an appreciation of the 
nuances of combat which can lead to the development of an 
ill-structured model. The military officer analyst 
represents a step towari providing a modeler or model 
operator who not only understands the mathematical aspects 
but the military factors as well. [If this capability is to 
be maximized when dealing with existing models, there must 
be documentation which allows the analyst to link the 
conceptual model formulation to the executable 
program. Then he can understand the conceptual basis of the 
model, insure the program fulfills the concept, and act as a 


source of information for the decision maker. 
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DST ORIGIN OF THE DOCUMENTATION SHORTFALL 


The initial theatre-level combat models used in the 
United States were designed and operated by Contract 
organizations (8.9. Research Analysis Corp.). These models 
had a manual referred to as a user's manual which provided 
an overview of the model, input requirements, and a 
description of the output. For an example see Ref. 22. 
Documentation within the manual was superficial, at most it 
provided the proverbial big picture Sufficient 
information was provided to determine the general 
capabilities of the model, possible suitability for a 
particular study and a general indication of expected 
outputs. It provided little if any information on the 
insights that could be attained or subtleties of the 
model. The user for whom this "user's manual" was designed 
was a project manager or some level of decision maker. The 
details could be filled in by the persons who would operate 
the model, this most likely was the designer of the 
model. Because of budgetary reasons and doubts that these 
model operaters understood the nuances of military 
operations, the decision was made to bring these models 


directly under Dept. of the Army control. 


When these models passed ta Army control all 
documentation passed with them. Unfortunately, in most 
cases the documentation was minimal consisting primarily of 
program listings, operator instructions, global variable and 
subroutine listings, flowcharts, user's manual and possibly 
some limited discussion of the formulation of the model. 
These models were placed under control of the agency that 
would use them in suaport of force planning and strategy 


studies. The designated users that inherited these models 
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were the military analysts (officers and federal employees) 
assigned to the organization that received the models. The 
analysts then took the model and the user's manual and 
proceeded to exercise the model in support of on-going 


projects. 


Use of the user's manual by the analyst was mandated by 
the fact that input requirements and preparation were 
contained in it. However, when the models were initially 
transferred from the designing corporation to the Army no 
supplemental documentation was prepared or 
provided. Analysts were uSing a manual that provided only a 
superficial examination of the model. They performed 
analyses using a manual originally designed for a study 
director or decision maker. These "user's manuals" did not 
contain the detailed information of the model that is 
necessary if the analyst is to know and understand the 
machinations through which the input data is exposed to 


produce a given set of outputs. 


Evenually conflicting results were obtained from models 
supposedly examining the same Situation. When called upon 
to resolve or explain these discrepancies the analysts were 
unable to do so. To explain the conflicts a detailed 
understanding of the conceptual basis of the model as well 
as detailed information of the translation of thea conceptual 
model into program code was needed. Only with this 
information could the analyst explain why two models 
examining the same combat process under the same conditions 


produced different results. 


Analysts then discovered that the documentation provided 
with the model did not provide this insightful 
information. To answer the question the model designer had 
to pe contacted. At times this was impossible because the 


actual model the designer no longer was with the firm that 
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developed the model, nor was there available any 
supplemental information on the inner workings of the model. 
Instances in which the designer could be located usually 
proved just as unrewarding. In most cases the designer had 
no supplemental formal documentation. When he acted as the 
Operator of the model on behalf of the Army he could answer 
such questions based on his design knowledge and daily 
contract with the model. Since there had been no formal 
reguirement for such supplemental information, it never was 
prepared. Once the model passed under operational control 
of the Army, the designer was precluded from đaily 
contact. Over time the designer's intimate familiarity with 
the model waned, especially the intracacies of the 
translation from basic concept to operating progran 
code. If the designer had not done the actually coding, 
similar results were usually experienced when trying to 
locate the original programmer or obtain supplemental 
information. What the program code was actually doing was 
not readily apparent and documentation or personal knowledge 
of its operation were unavailable. The situation is best 
expressed by a quote from Robert Frost. When once asked by 
a critic what he meant by a particular phrase in one of his 
works, he replied, "When I wrote it God and I knew what it 
meant, now only God knows". This same situation will also 
prevail with combat models unless adequate documentation is 
created concurrent with the design and development of the 


model. 


Er COMMUNICATION AND LEVELS OF PARTICIPATION 


The prime purpose of documentation is communication: 
communication of why and how realities are abstracted and 
condensed into a form suitable for exercise on a computer to 


predict a future state of some combat process. Generally 
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the level of professional communication between decision 
makers, analysts, and modelers has been low as attested by 
the credibility problem previously discussed. This coupled 
with the turnover of military decision makers and analysts 
has not enhanced couprehension of and control over 
large-scale combat models. Very often the information about 
context and limitations of these models has not been fully 
understood by the military analyst and therefore not clearly 
presented to the decision maker. Documentation that can 
provide such enlightenment to the analyst will he a 
Slgnificant step toward providing the required insights into 
combat models needed by decision makers. Through adequate 
documentation the user and analyst gain understanding, and 
with understanding can come acceptance and the decision to 


use the particular model. 


In combat modeling there are three levels of 
participation with necessary intercommunication. At the 
highest level is the decision maker who uses the model as an 
add” to gain insights and in conjunction with judgement 
arrive at a decision concerning force structure OT 
strategy. The intermediate level is occupied by the 
analyst, who either designs a model or useS an existing 
model; prepares inputs; exercises the model and interprets 
the outputs. If the model is new to the decision maker he 
also aids in the model selection process by recommending and 
explaining the inherent attributes of particular models in 
pursuit of a projected study. At the lowest level is the 
programmer, who writes the necessary code during model 
construction, explains the limitations of execution of a 
model on a particular computer, insures the model is 
processed as the program intended it and to code necessary 
modifications to a nodel. Each level has unique 
requirements and responsibilities and therefore the 


documentation required by each is unique. 
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The decision maker uses his documentation to gain an 
overall appreciation of a nodel and sufficient understanding 
to question the analyst on model details necessary to make a 
decision to employ the model and use its outputs. It 
provides a means of conducting discussions that can reveal 
what insights can be gained from a particular model. The 
decision maker uses his documentation as a link between the 


analyst and himself. 


The analyst occupies the pivotal position between the 
decision maker and the programmer. His documentation is 
necessarily the most broad in scope as well as variation of 
detail. It must provide intimate knowledge of the model to 
allow selection, sufficient detail to answer questions from 
the decision maker and ask questions of the programmer. It 


must provide the key to the inner workings of the model. 


Programmer documentation allows the programmer to 
Maintain and modify th2 model and answer the questicns of 
the analyst. It allows the programmer to maintain 
efficiency of operation and insure execution in accordance 
With the dictates of the design. It must clearly indicate 
how the computer interacts the inputs and programming code 
to produce the outputs. 


D. A CONCEPTUAL THEORY OF MODEL DOCUMENTATION 


Current texts that describe the documentation process 
usually are texts on the subject of computer systems. The 
guidance provided, though attempting to be general, is 
Oriented toward documentation of programs that are bulk 
processors of systematic information, i.¢. financial 
records, administrative records etc. Most emphasis is 


placed on how to use tha available program to process the 
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data available, minimal emphasis is directed toward how the 
processing is accomplished, because the processing is not 
complex but routine and easily understood. For an example 
see Ref. 33. 


In the field of combat modeling little guidance is 
available in the form of textbooks describing documentation 
methodologies. In order to maximize detailed unlerstanding 
as well as transferability of combat models a new concept of 


documentation is proposed and set forth in this section. 


To pata 11 the requirements discussed above the 
documentation must provide the complete understanding of how 
the model was created, underlying assumptions, explicit 
description of the fornulations and numerical methods 
employed and detailed description of the mechanics of the 
program code and its execution. Most current documentation 
described as a user's manual (e.g. see Ref. 22), executive 
Summary or by some Similar title is sufficient to fulfill 
the proposed non-technical documentation that follows. 
Programmer's documentation is widely described in the 
literature of computer systems and needs just a few 
additions in respect to combat models. Assuming that the 
documentation is provided as described in the Literature, 
Minimal modification is required. Primarily this is a 
charting and cataloging procedure to allow ease of tracing 
Variables through the program and understanding the linking 
of subprograms to the main progran. Current documentation 
reguirements stress Flowcharting. Flowcharting 1s 
comparable to electronic schematics and allows one to see 
the logical flow of the process being programmed. In the 
coding step this logical flow is translated into a 
mechanical flow of variables through mathematical and 
logical formulations and subprograms. A charting procedure 
will be like a blueprint that shows the physical connection 


of the main and subprograms. The catalog will explicitly 
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state how variables are interacted and passed between 


subprograms. 


For the analyst's documentation a new concept is 
proposed. Rather than describing the model in the 
Baar aonal manner, i.e. from the context of justification, 
the analyst's documentation should be presented from the 
context of discovery. [2] Using the traditional concept the 
model is documented by stating the assumptions of premises 
Which determine the outputs of the model, showing the final 
Mathematical formulae that represent the abstractions of the 
relavent characteristics of the modeled process, discussing 
the inputs required to support the given formulae, and 
listing the outputs that can be obtained from the model and 


inputs. 


This is not the manner in which the model was formulated 
and such documentation in the context of justification does 
not enlighten the analyst in terms of how the designer 
arrived at this particular abstraction of the combat 
process. One must conclude that this type of documentation 
is not of great help to the inexperienced analyst if he is 
attempting to understand the essence of a combat model. If 
the analyst is to gain a intimate knowledge of the 
functioning of a model, hə must be able to ascertain how the 
model came to be. This provides him two invaluable 
insights. First, he gains an insight into the abstraction 
process of the designer. It is a glimpse of th® reasoning 
procsess by which the designer cut through the myriad of 
details to deduce the fundamental variables that allow a 
model to approximate the complexity of a given combat 
process without having to model each of the multitudinous 
factors that compose the actual process. Second, he gains 
knowledge of the factors that were considered as 
representative of the combat process but were discarded 


because they seemed not to be necessary predictors. This 
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ASA itself is worth the cost of providing this type of 
documentation. What it provides is a historic record of 
factors and related assumptions considered and reasons why 
they were ultimately rejected or accepted. The information 
provided must be succinct and allow understanding of the 
abstraction process without introducing voluminous detail 
and unnecessary costs. This will be an aid not only for 
the follow-on analyst but all those that come later. [It 
Will preclude subsequent analysts from expending time and 
money to determine why particular factors are or are not 
modeled. Hence, if they wish to modify or elaborate upon a 
combat model and information about previously considered and 
rejected alternatives is available, the expenditure of 
resources to reinvestigate a particular alternative and gain 


only duplicate information will be precluded. 


E. REQUIRED LEVELS OF DOCUMENTATION 


All the evidence gathered indicates three levels of 
documentation for models are required. These levels of 


documentation are: 


(1) decision-maker level, (facilitates communication 


between the decision maker and analyst); 


(2) analyst level (fosters communication between analyst 
and decision maker and enhances working relationship with 


the programmer); 
(3) programmer level (provides detailed understanding of 


program code and facilitates communication with the 


analyst). 
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First, the decision maker who uses the model as an aid 
to his judgement requires a non-technical description of the 
model. Non-technical not because the decision maker cannot 
comprehend the technical details but because the exigencies 
of the organization preclude him from devoting the necessary 
time and effort. Second, the analyst that exercises the 
model, prepares the analysis, and assists the decision maker 
requires a technical description of the model. This enables 
the analyst to know how the model functions and explain and 
interpret the outputs of the model. Third, the programmer 
needs detailed documentation that explains the mechanics of 
the program. This documentation provides the means to 
troubleshoot problens encountered during routine running of 
the model and implement future modifications. The levels 
of activity and their required documentation are shown at 


Erg 2. 


F. DECISION MAKER'S NON-TECHNICAL DOCUMENTATION 


A non-technical reference manual for use by decision 
makers should provide sufficient information to determine 
general applicability of the model. It should include a 
general description of how the model operates and the major 
components, the specific purpose for which the model was 
originally designed and the known limitations should be 
stated. This will allow the decision maker to assertain the 
overall ability of this particular model to assist him in 
the problem at hand. The manual should describe the data 
requirements, available outputs, and any options provided by 
the model. This facilitates the initial planning process 
because it provides a basis for estimating the expected time 
to gather the input data and what to expect in the form of 


Outputs. Physical limitations, such as computers for which 
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usable versions of the nodel are available, the programming 
language used, and typical running times for the model, are 
necessary contents of the non-technical manual. The time of 
such senior managers is limited and valuable. Only that 
information necessary to provide a general description and 
basis for meaningful discussions between the decision maker 
and the analyst should be included. Any details required 
«ill be provided by the analyst, who will conduct the study, 
during planning, progress, and review sessions. Yet the 
documentation must make the model sufficiently transparent 
so that the decision maker understands what is happening and 


finds the model credible for the problem at hand. 


G. ANALYST'S CONCEPTUAL-TECHNICAL DOCUMENTATION 


The analyst's conceptual-technical reference manual must 
necessarily contain detailed information on all aspects of 
the model. This is the document that determines the 
overall worth of the model as an analytical tool. If 
sufficient detail is contained then the analyst can become 
intimately familiar with the model, so that any aberation 
encountered during operation of the model can be understood 
by him and explained to the decision maker. To maximize its 
value to an analyst, it should be written by the analyst or 
analysts that design the model. Their professional 
experience will guide them in providing the kind of 
information about the nodel that they would want if they 


were using a model designed by someone else. 


The size of the analyst's reference manual will be a 
function of the complexity of the model. To be useful and 
credible a model must be transparent and transferable. To 
be transparent and transferable all necessary details of the 


model must be provided. Information on the data base used 
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in the model, input requirements and format as well as 
output format and options must be detailed. All that can be 
assumed is that the analyst is knowledgeable in the use of 
the tools of his profession. In preparing the analyst's 
manual no prior knowleige of the model can be assumed. As 
Morris {2] recommended, the obvious should be written down. 

All constraints and limitations must be described in detail 
as well as assumptions used, logic flow and 
interactions. Sufficient technical detail to allow the 
analyst to manually trace inputs through the algorithms is 
necessary. Mathematical, Statistical, ani numerical 
methods incorporated in the model should be described 
including any new or unique applications. Any constraints 
Which will affect the accuracy of the model must be 
identified. Obvious pitfalls must be stated; they are only 
Obvious to the developer and in complex models without 
documentation they ean even be forgotten by the 
developer. The physical processes Simulated must be 
described including an explanation and rationalization of 
the techniques used. Each variable and the entity it 
represents must be clearly stated. Lastly, sufficient 
instructions describing how to set up and use the model and 
flow charts keyed to the program instructions should be 
provided. Equally important is a system that keys the 
description of each mathematical formulation in the manual 
to the appropriate section and lines of the program 
code. This will facilitate the location of th= code, when 


the analyst wants to modify the model. 


These are the basic requirements for the contents of 
documents to be prepared by the designer of a model. Each 
time a modification to the model is made, the changed 
program instruction, reasons why the change was necessary, 
how and what is affected in the model, and the 
mieaonalization for the particular modification should be 


documented and incorporated into the original 
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documentation. This asp2ct of documentation is critical 
because analyst and programmers have been known to make 
small insignificant changes to programs without documenting 
them. Two results can come about because of this. Over 
time these minor undocumented changes accumulate, then one 
day those who implemented the changes transfer or depart. 

The newly arrived analyst now has a model which does not 
quite fit his documentation. If enough time has passed 
even those who made the change will not completely remember 
it when they are questioned or they are no longer 
available. Even more disheartening is that in failing to 
document, a critical inspection of the effect of the change 
on other parts of the model is not made. Unbeknownst to the 
individual the modification causes an effect elsewhere in 
the model that is not readily. discernable at the 
time. Utimately, another modification is made and the model 
malfunctions or an anomalous output occurs. If undocumented 
modifications have been made and forgotten it may be 
impossible to trace the cause of the anomaly. If it can be 
traced, the cost of trouble shooting and correcting the 
model will be greater than the cost of documenting the 
modifications at the time of their addition. Cur bent 
studies will be delayed with their attendant costs and the 
credibility of the modeling community will suffer. The 
decision maker justifiably becomes skeptical when the 
analyst cannot explain why the results of a model are not 
compatible with the decision maker's intuitive 
expectations. If a model is to aid in the decision making 
process its outputs must be explainable and understandable 
to the decision maker. If the analyst has not d2signed the 
model then his only source of the necessary information is 


the documentation supplied with the model. 
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B. PROGRAMMER'S TECHNICAL DOCUMENTATION 


The final level of required documentation is the 
detailed information required by the programmer for the 
operation and maintenance of the model. This is the 
documentation that is discussed in great detail in all 
standard texts, while methods of documenting the conceptual 
basis of models is almost entirely foregone. The manual is 
necessarily prepared by the model programmer in conjunction 
With the analyst who has designed the model. It must 
contain descriptions of each global variable and in which 
subprograms the variable is found. Descriptions of the 
functions performed by the program, data flow charts, 
function flow Charts, approximations and numerical 
procedures used are also contained in this 
documentation. Likewise, any implicit assumptions made by 
the programmer during the coding process to facilitate 


computation must be documented. 


Often, though the analyst designer has structured the 
model to handie the general case, during implementation of 
the model on the computer some loss of generalization 
Occurs. in the search for programming efficiency the 
computer programmer may code a combat process so that the 
model works for only those circumstance explicitly stated by 
the designer. Whereas the designer believes he has 4a 
general model the programmer has reduced the generality of 
the model. Unless this is documented by the programmer, in 
subsequent use of the model the fact that the nodel was 
programmed in such a manner may be forgotten. In this case 
the computer routines embody more assumptions than exist in 
the formal model designed by the analyst. It is incumbent 


upon the programmer to bring this fact to the attention of 
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the analyst. He inturn nust include this information in his 
documentation. Only he can conceptualize in terms of real 
world factors the impact of the programmer's assumptions on 
the inputs and outputs. 


Further contents of the detailed programming nanual must 
indicate primary and secondary storage requirements, error 
detection and recovery procedures, and instructions for 
initiation and termination of program operation. These 
pieces of information are required if other programmers are 
to be able to understand and operate the model. If a model 
is transferred this infornation is necessary to implement 
the model on a computer of different design. It provides 
the receiving programmer the necessary information to 
prepare system operating programs and procedures so that the 
model can be exercised by the receiving agency without 
inadvertently making changes to the logic orf the 
model. This is especially cTit cal FOr complex 
models. With such models attempts to restructure programs 
to overcome incompatibilities in storage or other machine 
requirements may introduce changes to the fundamental logic 
of the model unbeknownst to either the programmer or the 
analyst. Without adequate documentation such restructuring 
must be accomplished with only luck to guide the way and as 
complexity of the model increases luck is quickly 
depleted. 


When the original programmer completes the programming 
(coding) of the model the model must be verified and 
validated in conjuntion with the analyst designer. When 
both are satisfied that the model is operating as designed a 
listing of the compiled assembled program should be 
made. This is then supplemented by a set of working notes 
on program operations, set up of the deck to exercise the 
model, special programming features and identification of 


potential problem areas and suggest2d solutions. 
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The compilation of the above documentation into a 
detailed programmer's manual will provide a sound basis for 
review and analysis of the model's capabilities and for 
Maintenance Ore gence model's “logic; at will facilitate 
transfer and insure transparency. Furthermore, each time 
the programmer makes a modification to the program it must 
be documented and incorporated into the programmer's 
Manual. Concurrently, it must be brought to the attention 
of the analyst to relate the programming change to the 
combat process modeled and update the analyst's and decision 
maker's manual. This coordination between the programmer 
and the analyst must always occur if both are to remain 
fully cognizant of what as well as how the model simulates 
the real world. Only through such documentation can a model 
be transferred to a new programmer and analyst team and be 


knowledgeably used. 


I. INLINE DOCUMENTATION, A MAP THRU THE MAZE 


A necessary adjunct to this proposed documentation 
package is documentation within the program. Such 
documentation should be in the form of comment cards that 
briefly describe the main program, the major sub-models, 
Subroutine and each function subprogram. They should be 
inserted at appropriate locations so that logic of the 
program is readily apparent. Nothing is more frustrating 
to an analyst that is attempting to gain a quick basic 
understanding of a model than a series of calis to 
Subroutines which inturn call other subroutines and or 
function subprograms. In a complex model this quickly hides 
the logic of the model from the analyst. To pierce this 
shroud the analyst must devote time that could better be 
used elsewhere. If the exigencies of the situation demand a 


rapid response and no other model is readily available this 
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encourages the analyst to use the model as the proverbial 
"black box". This neither adds to transparency nor does it 
enhance the reputation of the analyst or operations 
research. 


Most current programming texts recommend using comment 
cards to enhance understanding of programs. In complex 
models this is not a nicety but a necessity. Time devoted 
to this effort initially not only enhances the use of a 
complex model but reduces recurring costs. Professionalism 
demands that a model should not be used without 
understanding how it functions or at least believing one 
understands how it functions. The realities of military 
personnel policies dictate that military analysts will 
rotate rapidly through assignments. If a program is not 
documented internally this self-educating step occurs each 
time a new analyst uses a particular model. Given a complex 
model and normal personnel turnover a sizeable cost is 
incurred. With adequate internal documentation this cost 
Will occur only once. Because of personnel rotation models 
not currentiy having such internal documentation should have 
arc added. Though it will detract an analyst for an 
additional amount of time initially, over the longer term a 
savings will be realized. Each subsequent analyst will not 
have to start from scratch when trying to gain an 
understanding of how the model functions; understanding will 


be facilitated by internal documentation via comment cards. 


J. UPDATING A MODEL'S DICUMENTATION 


Changes to models occur over extended periods of time. 
Obviously, at the time changes are instituted the process 
through which they were developed, why and how they were 


entered into the model and all assumptions used nust be 
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annotated in all appropriate documentation at the agency 
instituting the change. Some of these changes will be 
extremely large and will justify formal changes being 
printed and distributed for all affected 
documentation. Some changes will be small; formal printing 
of individual changes would not be justified. These would 
have to be accumulated and changes printed, when 
economically practical. The management of the process and 
the determination of when printing and distribution of 
changes iS economically practical are beyond the scope of 
the thesis. They are important subjects and should be 


examined in some future thesis. 


K. SUMMARY 


Since the analysts and programmers that develop a 
particular model are not always present or available when 
the model is exercised, especially if it is transferred to 
another agency, it is their responsibility to detail their 
assumptions, simplifications and methodologies and provide 
evidence that the rationale behind their approach will 
produce results usable in the real-world environment as 
viewed by the ultimate decision maker. Analysts and 
programmers that create a model must provide documentation 
that establishes the issueS examined by the model, 
underlying the objectives and assumptions, the usability and 
usefulness of the model. Only with such documentation can 
the analyst or analysts assisting 2 decision maker in the 
resolution of a particular problem conclude that the use of 
a specific model is appropriate. The real "user" of a 
model is the decision maker. The analyst must recognize 
in assisting the decision maker he is a model designer and 
model manipulator. The officer analyst must gain detailed 


knowledge of any model he uses in support of an analysis. 
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To acquire such knowledge a model must be adequately 
documented. In combat modeling there are three levels of 
participation each with unique documentation requirements. 
These levels are: 

a Decision-maker level; 

s Analyst level; 


e Programmer level. 


TO enhance the ability of the analyst to fully 
understand the model his documentation should be presented 
from the context of discovery rather than the traditional 
Bomcest OL justification. This will allow the analyst to 
know the alternatives considered and rejected in structuring 
a model as well as giving him greater insights into the 


underlying hypothesis of a model. 


The types of documentation required are: 
s Decision Maker's Non-technical Documentation; 
s Analyst's Conceptual-technical Documentation; 
s Programmer's Technical Documentation; 


a Inline Documentation. 


Since model modification and improvement is 2 continuing 
process the preparation of formal changes must be 
economically practical and carefully managed. This area of 


model documentation is suitable for a subsequent thesis. 
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VII. CONCLUSIONS, 


A. SUMMARY OF PROBLEMS DISCUSSED 


Since 1973 the impetus in the Army modeling community 
has been to develop scenarios and models that can be used 
throughout the Army community for decisions on material 
requirements and force development. The ultimate purpose is 
to bring consistancy into the decision making 
process. Inherent in this qoal is not only the need for 
standard agreed upon scenarios but for a repertory of models 
to be used by all interested agencies examining 2 particular 
facet of the Army. This loes not necessarily mean that only 
one nodel should be used for a particular 
investigation. Because of the assumptions necessary to 
develop any model, given the necessary time and money it is 
best to exercise more than one nodel in order to get an 
indication of how the assumptions affect the outputs of each 
of the models. What is fundamental is the need for all 
agencies examining a given situation to be able to 
understand each other's nodels; this will provide the basis 
for intelligent discussion of the pros and cons of equipment 


and force requirements. 


AS operations research and system analysis have gained 
acceptance by DOD and Department of the Army, more combat 
models of various levels of military operations have been 
created and used. Increased use of models resulted in 
increased levels of complexity. Complexity in turn caused 


new models to be created when potential users could not 
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sufficiently understand and use existing models given 
available documentation. This unnecessary proliferation 
resulted in unexplainable conflicting results which caused 
the credibility of combat modeling to be questioned. Le 
this situation is to be resolved unnecessary proliferation 
must be checked and mutual understanding enhanced. To 
achieve this goal models must be easily transferable between 
investigating agencies. Fundamental to this ability is 
complete and up to date documentation of the model to be 
reviewed or exercised. This documentation will allow the 
potential user to analyze the model and determine its 


Suitability for a given project. 


Concomitant with the need to easily transfer models is 
the need to fully understand the machination by which a 
given set of input data is acted upon to produce 
outputs. If outputs of a computer model arae tə be useful 
they must be credible. Conflict roges outputs ot? military 
models are consistently challenged by some government agency 
of the DOD, the Congress, the Executive Office or some other 
branch of the government. Unless these challenges are 
answered and conflicts explained, the credibility of combat 
modeling will continue to suffer. Some criticism is needed 
to purify combat modeling and identify errors that 
inevitably will appear, some eminates from those advocating 
a competitive model or concept and some of it is from those 
that criticize the validity and usefulness of combat 
modeling itself. At times inadequate documentation has 
precluded the explanation of conflicting results by the 


analyst and added to the criticism. 


The Comptroller General [15] found that existing models 
have been used by other than the designer without thoroughly 
understanding their implications and limitations. At times 
this has resulted in erroneous conclusions being drawn and 


decisions made based on these conclusions. Subsequently, 
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the errors are surfaced with a loss not only in dollars 
expended in pursuit of undesirable projects but in further 


loss of credibility for theatre-level combat modeling. 


Adequate documentation will help alleviate some of the 
adverse publicity and loss of credibility that theatre-level 
combat modeling has experienced in the past. To a great 
extent this has stemmed from unexplainable contradictory 
results using models purporting to represent the same 
process. Mom dlilay the Criticism of models and their 
outputs and to enable both the analyst that exercises the 
model and the decision maker that uses the outputs to aid in 
the decision making process the nodel must be 
understood. Understanding can be enhanced through proper 
documentation. Adequate documentation of the form proposed 
Will facilitate communication between the decision maker and 
analyst as well as between the analyst and the 
programmer. Unless thase communication links are 
established misunderstanding of combat models will persist 
and the credibility of combat models and modeling will 


suffer accordingly. 


B. CONCLUSIONS AND RECOMMENDATIONS 


The conclusions drawn from this research are: 

a The decision maker is the "user" of a model. 

a To be of value a model must be accepted by the 
decision maker. 

a The analyst must relate the abstractions of the 
model to the actual combat process. 

a The analyst is a model manipulator. 

«a The analyst must understand and explain the 
methodology and structure of a model. 

a Understanding is deterred by complexity. 
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Models are becoming more complex. 

Increased levels om complexity result in 
diminished transparency. 

Inability to understand and use existing model 
causes development of redundant models. 
Unnecessary proliferation causes conflicting 
results to be produced by similar models. 
Inability to explain conflicting results is the 


basic cause of a lack of credibility. 


Frolsjferation can be ređuced through model 


Sharing. 


Prior 


Sharing a model requires: 


ability to use it with minimal conversion; 
adequate facilities to run the model; 

sufficient competent analysts and programmers; 
adequate documentaton; 

formalized arrangements for cost sharing and 


coordination. 


to transferring a model it must be analyzeđ for 


applicability and appropriateness. Such an analysis should 


consider: 


Many 


inputs and outputs; 

basic causal relationships; 

model logic and structure; 
available data and required data; 


time and resources required. 


existing Combat nodels are characterized by: 


Submodel inadequacy; 

key variables excluded; 
inherent inaccuracies; 
eompucational difficulties; 
inconsistent assumptions; 
inappropriate assumptions; 


inadequate? output. 
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The degree to which meaningful model evaluation can be 
accomplished is significantly influenced by available 
documentation. Documentation is key to the understanding 
of complex models. The research conducted indicated: 

a Adequate documentation is a necessary if a 
model is to be used by other than the 
originator. 

= Organization exigencies deter adequate 
documentation. 

a Documentation of combat models is generally 
inadequate. 

a If a model is poorly documented it may be more 
economical to build a new one than share an 
existing mod2l. 


e Efforts are being made to improve documentation. 


The author's experience with and research of combat 
model documentation indicates that there are three levels of 
interaction with combat models. These levels have unique 
and common requirements for documentation. To satisfy these 
requirements the author envisions three levels OF 
documentation: 

eee DECISTON MAKER'S NON=TECHNECAL 
a ANALYST'S CONCEPTUAL-TECHNICAL 
a PROGRAMMER'S TECHNICAL 


The decision maker's and the programmer's documentation 
must provide the information listed in chapter six. It can 
be presented in the traditional manner using techniques 
contained in most computer system management texts. 
However, to function as the link between the decision maker 
and the programmer and to understand the nuances of the 
model, the analyst needs documentation that provides greater 
insights than possible with the current available 


documentation. These insights will be provided if the 
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analyst's documentation is presented 
the 


discovery rather than 


justification. 

Many changes 
time. 
formal 


practical Con print 


control, and management is 


hierarchy of documentation. 


for future research. 


Co FINAL REMARKS 


Acceptance or rejection 


The method of determining 


the 
traditional 


fron context of 


contert of 


to a model occur over extended periods of 


when it is economically 


their disribution, 
the 


topics are appropriate 


Changes, 
erieical, to proposed 


These 


of an expository thesis in 


matters such as documentation often depends on th2 skill of 


the pleader and 


Same set of evidence the parties to the debate can 


Sharply different 


notions may lead them to select and interpret the 


in different ways. Even 


the mood of the audience. 


conclusions, 


Staring at the 
come to 
their 


Since preconceived 


evidence 


though one may initially find it 


difficult to believe that there are ways to acquire adequate 


documentation not yet 


agencies researching the problem, the 


pervasiveness of the 


combining the various proposals in different 


sone combination will 


proposal presented is but one possible means to achieve 


desired end. 


tried 


problenm 


produce 


by analysts or advocated by 


very complexity and 


suggests the possibility of 
ways so that 

the desired goal. The 
the 


Of even moce importance is the fact that some 


methodology must be adopted to correct this lack of adequate 


documentation. 


the problem has persistei for almost twenty years. 


Bas it made it 


between interested agencies but it 


analysts from gaining full 


With regard to theatre-level combat models, 


Not only 


near impossible to easily transfer models 


has preventei military 


knowledge of the nodels they 





use. 


The conflicting opinions and evaluations unmasked during 
this research confirm what is intuitively obvious: many of 
the historical judgements and decisions concerning 
operations researc in general and theatre-level combat 
modeling in particular are based on subjective values as 
well as objective facts. Unlike the natural scientist or 
the analyst using a simulation, the researcher examining the 
process through which combat modeling has evolved, cannot 
reproduce the events and by experimentally altering the 
ingredients, change the result. The development of combat 
modeling is well documented; yet controversies have 
developed despite the voluminous sources. Analysts disagree 
not because one may be more knowledgeable about the subject 
than another, but because each weighs and evaluates 
differently those facts of which both have knowledge. There 
is little dispute about the details of what has happened in 
the development or theatre-level combat models but there is 
intense disagreement on the significance of past events and 
how to proceed in the future. The analyst has no fixed 
point from which to observe the stream of events concerning 
the development of theatre-level combat models. Analysts 
are borne along by the current and their interpretation of 
what has occurred is influenced by their view of where the 
stream seems to be headed and whether the apparent 
destination appears to be good or bad for the enhancement 


and development of the OR profession. 


Although theatre-level combat modeling attempts to be 
scientific in its methods, it is rarely so in its 
outputs. Outputs are interwoven with subjective judgements, 
either through their interpretation or by way of the inputs 
that were instrumental to producing the outputs or in the 
Very construction of the nodel itself. The relativity of 


subjective judgement, while discouraging, need not be 
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debilitating to theatre-level combat modeling. Because 
these models are the only means of examining force structure 
questions vital to national security, analysts are 
confronted with the continuing task əf rethinking the basic 
structures of these models. Past models can furnish us a 
vast reservoir of experience in theatre-level combat 
modeling which can be exploited to further this aspect of 
operations research. However, this reservoir can be 
effectively used for the enhancement of the profession only 
if what has been accomplished is adequately documented so 
that others may correctly use models previously 
developed. In this manner, even though no model can fully 
treat all the intricacies of the combat process, the analyst 
can enrich the profession through the continuing effort to 
better model the process fully uSing all the knowledge that 


has come before. 


The concept of documentation that this paper proposes 
Meeeeenot cure all the ills. It is but a proposal to correct 
a defect, inadequate documentation, that has long plaqued 
theatre-level combat modeling. Bute e@ee 2t is faithfully 
executed with the same energy and level of effort that has 
been expended in decrying the problem of inadequate 
documentation, then there is hope that th2 omission can be 
corrected. It is imperative that analysts adequately 
document newly designed models or modifications to existing 
models so that other analysts may use them properly. Before 
a project is considered complete it is the analyst's 
responsibility to insure that the vital step of 
documentation is accomplished. Oniy in this nanner can 
there be assurance that the model designed or modified can 
be fully understood by those who subsegquently want to use 
the model. Analysts using existing models must expand and 
supplement the current available documentation of existing 
models in the active inventory. The next time an existing 


model is used professionalisn demands atthe be fully 
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understood; any new undocumented factors uncovered in its 
examination should be formally noted and made a permanent 
part of the official documentation. In this manner past 
omissions will be correctad, the scientific method will be 
invigorated and the standing of the Operations Research 
profession enhanced. Subsequent results will then be more 
fully explainable and posses greater credibility and even if 
the conclusions cannot be final, because of the impalpable 
nature of the subject, the techniques of theatre-level 


combat modeling will be enriched by the process. 
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